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(54) Router line card protection using One-for-N redundancy 



(57) Router line cards are partitioned, separating 
packet forwarding from external or internal interfaces 
and enabling multiple line cards to access any set of ex- 
ternal or internal data paths. Any failed working line card 
can be switchably replaced by another line card. In par- 
ticular, a serial bus structure on the interface side inter- 
connects any interface port within a protection group 
with a protect line card for that group. Incremental ca- 
pacity allows the protect line card to perform packet for- 
ward functions. Logical mapping of line card addressing 
and identification provides locally managed protection 
switching of a line card that is transparent to other router 
line cards and to all peer routers. One-for-N protection 
ratios, where N is some integer greater than two, can 
be achieved economically, yet provide sufficient capac- 



ity with acceptable protection switch time under 1 00 mil- 
liseconds. Alternatively, protect line cards can routinely 
carry low priority traffic that is interruptible, allowing the 
protect line card to handle higher priority traffic previ- 
ously carried by a failed working line card. This ap- 
proach renders unnecessary engineering a network for 
less than full capacity to allow rerouting in the event of 
individual line card failure. Consequently, all data paths 
can be fully utilized. If a particular interface module on 
one data bus needs removal for maintenance, a dupli- 
cate data bus is available intact, allowing hot replace- 
ment of any working or protect interface module, even 
while a line card protection switch is in effect. 
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Description 

RELATED APPLICATIONS 

[0001] This application is related to concurrently filed, 
co-pending, and commonly assigned U.S. Application 
Serial Number 09/703,057, entitled "System And Meth- 
od For IP Router With an Optical Core," to concurrently 
filed, co-pending, and commonly assigned U.S. Appli- 
cation Serial Number 09/703,056, entitled "System and 
Method for Router Central Arbitration," to concurrently 
filed, co-pending, and commonly assigned U.S. Appli- 
cation Serial Number 09/703,038, entitled "System and 
Method for Router Data Aggregation and Delivery" to 
concurrently filed., co-pending, and commonly assigned 
U.S. Application Serial Number 09/702,958, entitled 
"Timing and Synchronization for an IP Router Using an 
Optical Switch," to concurrently filed, co-pending, and 
commonly assigned U.S. Application Serial Number 
09/703,027, entitled "Router Network Protection Using 
Multiple Facility Interfaces," and to concurrently filed, 
co-pending, and commonly assigned U.S. Application 
Serial Number 09/703,064, entitled "Router Switch Fab- 
ric Protection Using Forward Error Correction," the dis- 
closures of which are incorporated herein by reference. 

TECHNICAL FIELD 

[0002] This application relates to the field of optical 
communication networks, and particularly to large-scale 
routers for optical communication networks. 

BACKGROUND 

[0003] Routers form a central part of a data commu- 
nication network and perform general routing. There can 
be multiple routers in a network. Information typically 
travels from one router to the next router, and eventually 
reaches the destination edge of the network. A destina- 
tion edge router receives the information and decides 
where it goes from there. Typically it goes to an Internet 
service provider at the opposite edge of the edge router. 
If the destination is a household PC, the Internet service 
provider then sends the information to the destination 
computer. If there is corporate access to the network, 
the information may go from the edge router directly to 
a corporate site. 

[0004] A fabric is a collection of devices which coop- 
eratively provides a general routing capability. Internet 
protocol (IP) routers require protection from fabric fail- 
ures, for example optical fabric, packet fabric, and 
switch element fabric failures. The prior art uses dupli- 
cated switch fabrics and line cards that feed both switch 
fabrics simultaneously but receive from only one switch 
fabric at any given time. 

[0005] Internet protocol (IP) routers are not protected 
from line card failures with hot standby immediate acting 
protection mechanisms. Current designs depend on the 



external rerouting of IP packets and flows to restore 
packet traffic around failed line cards. This mode of pro- 
tection is slow and is cumbersome to engineer and ad- 
minister. A particular problem is that, in the event of fail- 

5 ures of line cards or packet forwarding elements, it is 
impossible to limit the effects of those failures to the rout- 
er in which the failure occurs. The downstream and up- 
stream peer routers have to change their routing tables 
and change their packet destinations and flows in order 

10 to reroute packets around the failed packet forwarding 
line card. 

[0006] An alternative approach is to implement multi- 
ple packet forwarding line cards to provide redundancy. 
This approach, however is economically unattractive, in 

*5 that it consumes multiple switch fabric ports, thus dou- 
bling the required port count of the switch fabric. This 
results inevitably in underutilizing any particular line 
card. In order for additional packet traffic to be rerouted 
onto a line card M in the event of failure of line card N, 

20 a network must be engineered such that line card M is 
operating continuously at less than its maximum capac- 
ity. 

[0007] Without fast acting hot standby protection, a 
network must be engineered with duplex and multiple 

25 routers and with less than fully utilized traffic capacity 
on each port. Then in the event of a facility or port failure 
during operation, all traffic must be redirected from the 
failed port to another port, which is available but under- 
utilized and which has enough intrinsic capacity to carry 

30 the additional traffic under such a failure circumstance. 
[0008] The first problem is not what happens once the 
failure occurs, but the way the network must be engi- 
neered to provide this complex protection structure. 
Once duplex routers or multiple routers are engineered 

35 into the network to address this type of failure, then typ- 
ically it is required to engineer additional line capacity 
into the network between those routers. Whereas an un- 
protected network might require only a single trunk that 
is 100% utilized between two routers, a protected net- 

40 work under current technology requires a second trunk. 
The utilization of each one of the trunks in the absence 
of failure falls to only 50%. This increases the cost not 
only of the equipment, but of the router itself that now 
includes redundancy, software costs relating to the in- 

45 tervening network capacity, fiber optic transmission ca- 
pacity including increased overhead traffic between 
routers . and administrative and engineering effort. 
[0009] In prior art schemes an internal failure within a 
router would have to be protected by rerouting of the 

50 trunk outside of that router, perhaps encompassing sev- 
eral other routers in an existing network. Failure of a ca- 
ble at a router can in fact propagate significantly far 
through a network, resulting in substantial confusion to 
the network as it adjusts to reconfigured routing. The 

55 network must broadcast to much of the Internet any IP 
addresses, for example, that have changed. Thus, small 
localized failures produce impacts that ripple out 
through the network, even though their original cause 
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may not have been significant. 
[0010] Not only do the packets get re-routed, but there 
is of necessity broadcast information that has to be sent 
to various routers to handle the re-routed traffic. In situ- 
ations where outages occur from time to time, this can 
become overwhelming to a network. Even in the best 
case, the time to perform a repair and restore the original 
configuration can cause network traffic to slow dramat- 
ically. Again, this affects the capacity of a network, which 
in the initial stage would have to be engineered for high- 
er capacity than would otherwise be necessary. 
[0011] A common problem is an intermittent fault in a 
network, coming into and going out of service repetitive- 
ly, thereby causing the generation of rerouting messag- 
es almost continuously through the network, known in 
the industry as ,, route-flap, ,, resulting in much non-useful 
traffic. 

[0012] Consequently, there is a need in the optical 
network art for router systems and methods that provide 
protection in the event of a failure, requiring a smaller 
investment in equipment and engineering effort than in 
the prior art. Further, there is a need for router failure 
protection that requires minimal disruption and recon- 
figuration of the larger network, and that provides seam- 
less continuity of service in the event of a single point 
of failure. 

SUMMARY OF THE INVENTION 

[0013] The present invention is directed to a system 
and method which partition the router line cards, thereby 
separating the packet forwarding functions from the 
physical port interfaces or facility modules and also sep- 
arating the packet forwarding functions from any internal 
router fabric interfaces. This enables multiple line cards 
to access any particular set of external facility or internal 
fabric paths. A method in accordance with the present 
invention further provides data and control paths that al- 
low any failed working line card within a protection group 
to be switchably replaced by another line card, which is 
provided exclusively for protection purposes within the 
protection group. In particular, a serial bus structure on 
the port side of a line card allows any optical port within 
a given protection group to access the protection line 
card for that group. Incremental excess capacity across 
the router fabric is provided, so that the protection line 
card can request and receive grants to transmit packets 
to the fabric. Logical mapping of line card addressing 
and identification is used, such that a protection switch 
of a line card is managed locally and is transparent to 
other line cards in the router and to all external peer rout- 
ers. 

[001 4] A benefit of this approach is that one for N pro- 
tection ratios of the line cards, where N is some integer 
greater than two, can be achieved, which are very eco- 
nomical, yet provide sufficient system and network 
availability with acceptable protection switch time per- 
formance. An attractive protection switch time is gener- 
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ally any time under 100 milliseconds. 
[0015] In an alternate embodiment, protection line 
cards can be used routinely for low priority traffic in the 
absence of failure of the working line cards. This low 
5 priority traffic can be interrupted to allow the protection 
line card to switch over to handle higher priority traffic 
previously carried by a failed working line card. In this 
approach it is not necessary to engineer network links 
of less than full capacity to allow for rerouting in the 
10 event of individual line card failure. Consequently, all 
ports can be used to full capacity. 
[0016] If a particular facility module needs to be re- 
moved for maintenance purposes on one data bus, the 
duplicate data bus is maintained intact, allowing for hot 
15 replacement of any of the facility modules, working and 
protect, even if a packet forwarding module protection 
switch is in effect at the time. 

[0017] Embodiments according to the present inven- 
tion are designed to protect against all single fault oc- 
currences. Single faults include a single fault of a mod- 
ule, a single fault of a cable, or a single fault of a path. 
Accordingly, although some double faults are protected 
against, double faults generally lie beyond the scope of 
primary objects of the present invention and thus are not 
in general protected against. 

[0018] Various aspects of the invention are described 
in concurrently filed, co-pending, and commonly as- 
signed U.S. Application Serial Number 09/703,057, en- 
titled "System And Method For I P Router With an Optical 
Core," concurrently filed, co-pending, and commonly 
assigned U.S. Application Serial Number 09/703,056, 
entitled "System and Method for Router Central Arbitra- 
tion," concurrently filed, co-pending, and commonly as- 
signed U.S. Application Serial Number 09/703,038, en- 
titled "System and Method for Router Data Aggregation 
and Delivery," concurrently filed, co-pending, and com- 
monly assigned U.S. Application Serial Number 
09/702,958, entitled "Timing and Synchronization for an 
IP Router Using an Optical Switch," concurrently filed, 
co-pending, and commonly assigned U.S. Application 
Serial Number 09/703,027, entitled "Router Network 
Protection Using Multiple Facility Interfaces," and con- 
currently filed, co-pending, and commonly assigned U. 
S. Application Serial Number 09/703,064, entitled 
"Router Switch Fabric Protection Using Forward Error 
Correction," the disclosures of which are incorporated 
herein by reference. 

[0019] The foregoing has outlined rather broadly the 
features and technical advantages of the present inven- 
tion in order that the detailed description of the invention 
that follows may be better understood. Additional fea- 
tures and advantages of the invention will be described 
hereinafter which form the subject of the claims of the 
invention. It should be appreciated by those skilled in 
the art that the conception and specific embodiment dis- 
closed may be readily utilized as a basis for modifying 
or designing other structures for carrying out the same 
purposes of the present invention. It should also be re- 
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alized by those skilled in the art that such equivalent 
constructions do not depart from the spirit and scope of 
the invention as set forth in the appended claims. The 
novel features which are believed to be characteristic of 
the invention, both as to its organization and method of 5 
operation, together with further objects and advantages 
will be better understood from the following description 
when considered in connection with the accompanying 
figures. It is to be expressly understood, however, that 
each of the figures is provided for the purpose of ilius- 10 
tration and description only and is not intended as a def- 
inition of the limits of the present invention. 



DETAILED DESCRIPTION 



15 



20 



BRIEF DESCRIPTION OF THE DRAWING 

[0020] For a more complete understanding of the 
present invention, reference is now made to the follow- 
ing descriptions taken in conjunction with the accompa- 
nying drawing, in which: 

Figs. 1 A-1C form a schematic diagram showing an 
overview of the data paths through a router, in an 
embodiment of the present invention; 



Fig. 2 is a block diagram illustrating data flow 25 
through facility modules of a router in more detail; 

Fig. 3 is a block diagram illustrating information flow 
through a typical packet forwarding module; 

30 

Fig. 4 is a block diagram representing information 
flow through a typical internal optics module, ac- 
cording to an embodiment of the present invention; 

Figs. 5A and 5B are schematic diagrams illustrating 35 
the functioning of a facility ASIC in the normal work- 
ing mode and in the protection switch mode respec- 
tively; and 

Fig. 6 is a flow diagram outlining the steps involved 40 
in performing an automatic PFM protection switch, 
in accordance with an embodiment of the present 
invention. 



45 



[0021] Figs. 1 A-1 C form a schematic diagram show- 
ing an overview of the data paths through a router 10, 
in an embodiment of the present invention. For ease of 
understanding, Figs. 1A-1C are partitioned into three so 
sequentially adjacent panels. Fig. 1A-1C do not show 
how router system 1 0 is wired, but simply illustrates the 
flow of data. At the upper left portion of Fig. 1 A, an input 
1 01 -0 is a first SONET data channel, formatted as Pack- 
et-over-SONET in the present embodiment. Input 101-0 55 
includes two optical fibers, namely a working input fiber 
1 01 -0W and a protect input fiber 1 01 -OP Fibers 1 01 -0W, 
101 -OP carry duplicated information into router 10 from 



a peer source equipment e.g., another router or piece 
of SONET transmission equipment, compatible with the 
Packet-over-SON ET format. Protect and working facility 
module cards 11 -OP and 11 -0W independently receive 
duplicate input from respective optic fibers 101 -OP and 
101-0W and perform an integrity check on the informa- 
tion by computing SONET parity and SONET framing 
words to determine if the information is valid, and inde- 
pendently check SONET protection switching 'K' Bytes. 
Both facility modules 11 -0W and 11 -OP perform essen- 
tially identical functions on the information. Each facility 
module independently evaluates the SONET frame and 
determines whether the information contained on it is 
valid. Facility modules 11 -0W and 11 -OP then extract 
packets from their respective SONET frames and trans- 
fer those packets over a packet bus 1 03 to a packet for- 
warding module (PFM) 13-0. 

[0022] Working facility module 1 1 -0W and protect fa- 
cility module 11 -OP respectively provide duplicate input 
interfaces 103-OW and 103-OP to packet forwarding 
module 13-0. A system controller (not shown in Figs. 
1A-1C) examines the status of facility modules 11 -0W 
and 11 -OP and selects as the in-service facility module 
the one that is receiving properly framed and bit-error- 
free packets on its input, in accordance with the SONET 
standard or as selected by SONET 'K' Bytes. Should the 
information coming into a facility module, for example 
facility module 11 -OP, have bit errors or other defects, 
then facility module 1 1 -OP raises an alarm at the system 
controller (not shown in Figs. 1 A-1 C). The system con- 
troller then selects facility module 11 -0W as the source 
of input from that channel, and facility module 11-OW 
strips the packets out of the SONET framing overhead 
and transfers those raw packets over industry standard 
bus 1 03-OWto packet forwarding module 1 3-0. Typically 
facility modules 11-0Wand 11 -OP, along with packetfor- 
warding module 13-0, are contained in a line shelf, de- 
noted in Fig. 1Aas 1 /^ line shelf 142 (ingress) and de- 
scribed below in more detail. 

[0023] There are actually N+1 multiple packet for- 
warding modules 13-0 through 13-N. In Fig. 1A, N 
equals 4, providing for four working packet forwarding 
modules 13-0 through 13-3 and a fifth designated pro- 
tect packet forwarding module 1 3-4. In this case protect 
packet forwarding module 1 3-4 is a spare module avail- 
able to replace any working module 13-0 through 13-3. 
Should any one of working packet forwarding modules 
1 3-0 through 1 3-3 fail, then fifth packet forwarding mod- 
ule 13-4 can substitute for the failed packet forwarding 
module 13-0 through 13-3. This protection configuration 
is known as "one-for-four" protection. Similarly, on the 
output side of router 1 0 shown in the right side portion 
of Fig. 1C, packet forwarding modules 18-0 through 
18-3 are all working modules, and packet forwarding 
module 18-4 is a spare protect packet forwarding mod- 
ule available as a replacement in the event of failure of 
any working packet forwarding module 18-0 through 
1 8-3. Typically packet forwarding modules 1 8-0 through 
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1 8-4 are contained in a line shelf, denoted in Fig. 1 C as 
V?. line shelf 143 (egress) and described below in more 
detail. 

[0024] Protection works through a daisy-chain data 
bus 105 cascading from Channel 0 to Channel 1, to 
Channel 2, to Channel 3, and to Channel 4, linking fa- 
cility modules 11 -0W through 11 -4W. A duplicate data 
bus interconnects from Channel 4 up to Channel 0, link- 
ing facility modules 11 -4P through 11 -OP. If for example 
packet forwarding module 13-1 were to fail, then input 
facility modules 1 1 -1 P and 1 1 -1 W send their traffic down 
data bus 105 linking facility modules 11-2 and 11-3 to 
facility module 11-4, which then switches those inputs 
to protect packet forwarding module 13-4. Thus if one 
channel fails, traffic, instead of going through the failed 
channel, goes down data bus chain 105 to designated 
protect module 1 3-4. If a particularfacility module needs 
to be removed for maintenance purposes on one data 
bus, the duplicate data bus is maintained intact, allowing 
for hot replacement of any of the facility modules, work- 
ing and protect, even if a packet forwarding module pro- 
tection switch is in effect at the time. Similarly on the 
output side of router 10, output data is rerouted up a 
data bus chain 106 to Channel 1 and then out of router 
10. 

[0025] In operation, if PFM 13-1 fails, a microproces- 
sor in the line shelf containing the failed packet forward- 
ing module detects the failure, notices if the system is 
configured for one-for-four protection, and instructs 
switches on facility modules 1 1 -1 through 1 1 -4 to switch 
traffic that used to be in Channel 1 down to Channel 4. 
Channel 4 contains facility modules 11 -4P and 11 -4W 
on the input side and facility modules 1 2-4P and 1 2-4W 
on the output side respectively of router 1 0. These mod- 
ules are connected to optical inputs and outputs only 
when utilizing protect PFM 13-4 or 18-4 as a working 
module and not as protection for PFMs 13-0 through 
1 3-3 or 1 8-0 through 1 8-3. If PFM 1 3-4 or 1 8-4 is a work- 
ing module, then daisy chain bus 105, 106 is not utilized 
in any way, and there are simply 5 working inputs and 5 
working outputs. Accordingly, two modes of operation 
are available; namely one-for-N protection, for example 
one-for-four; or zero-for-five protection, meaning no 
protect modules and five working modules. Without re- 
quiring any wiring changes, router system 10 will func- 
tion in either mode. 

[0026] An alternative operating mode designates in- 
put 1 01 -N and output 1 02-N for lower priority traffic. That 
traffic would be deliberately interrupted in the event of 
a failure of any of the packet forwarding modules carry- 
ing higher priority traffic and requiring a protect packet 
forwarding module to service that failure. 
[0027] Information is transferred from PFM 1 3-0 to in- 
ternal optics modules (lOMs) 14 as chunk payloads of 
data, such that a chunk contains typically 400 bytes of 
payload data. Packets contained in virtual out queues 
of PFM 1 3-0 that are destined for the same egress PFM 
can be combined to form a single chunk payload of data. 



Thus, multiple small packets or just a segment of a larg- 
er packet can be loaded into a single chunk. A maximum 
of two chunks can be transferred from a PFM 13-0 to 
the lOMs 14-0W0 and 14-1 W0 during each chunk peri- 

5 od. The same chunks are replicated and transferred in 
parallel tolOMs14-0W1 and14-1W1. 
[0028] IOM modules 1 4 encapsulate FEC code words 
as multiple redundant check symbols into each of the 
chunks. The present implementation uses a convention- 

10 al interleaved Reed-Solomon FEC coding. IO modules 

1 4- 0W0, 1 4-1 W0 provide duplicate working module ca- 
pacity for a working zero optical switch plane. Similarly 
IO modules 14-0W1, 14-1W1 provide duplicate working 
module capacity for a working one optical switch plane. 

15 Switch plane pairs in this case are not configured as 
working and protect, but as working zero and working 
one copies respectively, such that copy zero switch 
plane containing optical switch modules 15-1 through 

15- 6 and duplicate copy one switch plane containing op- 
20 tical switch modules 16-1 through 16-6 each provide 6 

optical switches worth of capacity. 

[0029] IO module 14-0W0 transfers information from 

PFM 13-0 to one of three optical switch modules 15-1, 

1 5- 2 and 1 5-3. IO module 1 4-0WO sends the information 
25 to the appropriate optical switch module based on the 

decisions of the central arbiter module (not shown in the 
figures), described in U.S. Application Serial Number 
09/703,057, cited above. Illustratively, one input comes 
into an optical switch module and one output goes out 
30 from that same optical switch module. In an actual sys- 
tem, these inputs and outputs in fact provide connectiv- 
ity across router system 1 0. Fig. 1 B shows optical switch 
module 15-1 connected to an egress side internal optics 
module 1 7-0W0 through an output fiber 110-1. For Clar- 
is ity, six such optical switch modules 15-1 through 15-6 
are shown in the top portion of Fig. 1B. In fact, in one 
implementation each of these optical switch modules 
has 64 optical fibers in and 64 optical fibers out, with 
these 64 optical fiber pairs fanning out to a great many 
40 different line shelves. Different shelves have multiple 
fiber inputs and outputs. Six parallel optical switch mod- 
ules 15-1 through 1 5-6 provide 6 times the data capacity 
of a single switch module. Other embodiments can have 
for example, 36 of these modules rather than six. 
45 [0030] Chunks of information are sent individually 
through optical switch modules 15-1 through 15-N and 

16- 1 through 16-N and received by IO modules 17 on 
line shelves at the egress side of router 10. IO module 
17 checks the FEC check symbols to validate the accu- 

50 racy of the data bits within the chunk. It then removes 
the FEC check symbols and transfers the resulting 
chunk payloads to packet forwarding module 1 8-0, 18-1, 
18-2, 18-3, or 18-4 as appropriate for each destination 
address. Similarly, the working one optical switch plane 

55 containing optical switch modules 16-1 through 16-N 
does substantially the same thing in parallel. Thus, 
working zero and working one optical switch planes per- 
form this process duplicatively and in parallel. This al- 
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lows the packet forwarding modules on the egress side, 
such as PFM 1 8-0, to select those chunk payloads that 
are error free either from working zero or from working 
one optical switch plane on a chunk by chunk basis. If 
there is an error in an optical switch, then egress PFM 
modules 18-0 through 18-N can identify which working 
plane, zero or one, is accurate. Consequently errors in 
a switch are contained and do not ripple out through the 
network. 

[0031] If there are only a few bit errors going through 
a switch, those errors can be corrected in real time by 
FEC decoding in IO modules 17. If a path through a 
working zero optical switch fails completely, then a path 
through the working one optical plane can be utilized 
instead. Further, because each IO module 1 7 computes 
the corrupted bits and how many bits were corrected on 
every path of the system, IO modules 1 7 provide a de- 
tailed fault analysis not only of the failed fiber or optical 
switch plane, but even down to the level of an individual 
switch defect, which then can also be isolated. Impor- 
tantly, the data flowing across for example OS Module 
15-1 and the data flowing across OS Module 16-1 in the 
absence of failures in the system are identical, byte for 
byte. This provides a hot standby, chunk for chunk. 
[0032] After selecting error-free chunk payloads, 
packet forwarding modules 18-0 through 18-N then re- 
assemble the chunks into individual IP packets and for- 
ward those packets across interface links 104, as pre- 
viously described. 

[0033] In Figs. 1 A-1 C for the purpose of clarity, corre- 
sponding input and output functions are shown on sep- 
arate circuit cards in separate 14 line shelves 142 and 
143 respectively. In some embodiments corresponding 
input and output functions are combined on a single cir- 
cuit card in a single line shelf combining 14 line shelves 
142 and 143, thereby creating a folded configuration. 
For example, working input facility module. 11-0W and 
working output facility module 12-OW can be combined 
on a single physical printed circuit card with two optical 
connectors, one in and one out. Similarly protect input 
facility module 11 -OP and protect output facility module 
1 2-OP can be combined on a single physical circuit card 
with two optical connectors, one in and one out. Like- 
wise, input and output packet forwarding modules 13-0 
and 18-0 also can be combined on a single physical cir- 
cuit card in a single line shelf. In a folded configuration, 
if packet forwarding modules 13-0 and 18-0 share the 
same physical card, then there is a single card for Chan- 
nel 0, likewise a single card each for Channels 1 , 2, 3, 
and a fifth card for a Protect channel 4. Because there 
is a single physical card for input and output functions, 
then If a card fails, the protection ratio is equal for both 
input and output modules on that card. In some embod- 
iments internal optics modules 14-0 W0 and 17-0W0 
similarly share the same physical circuit card, which in 
the present implementation is contained in the same line 
shelf 142, 143 with combined input/output facility mod- 
ules 11 , 12 and combined input/output packet forward- 
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ing modules 13, 18. 

Fig. 2 is a block diagram illustrating data flow through 
facility modules 11-0Wand 12-OW, for example, in more 
detail. Facility optical fibers are connected on the left 

5 through input and output interfaces 101-0W and 
102-OW respectively. In a preferred embodiment shown 
in Fig. 2, for purposes of illustration input and output fa- 
cility modules 1 1 -0W and 1 2-0W occupy the same circuit 
board in the same line shelf in a folded configuration. In 

to other embodiments, the input and output facility mod- 
ules 1 1 -0W and 1 2-0 W are located on separate physical 
circuit cards. 

A signal, e.g., a packet-over-SONET (POS) formatted 
IP packet, arrives at input 1 01 -0W to a signal processing 
is module 201 typically in a ten-Gbit/sec OC192 SONET 
datastream. Processing module 201 contains an optical 
receiver, an optical multiplexer and associated demulti- 
plexer, and a transmitter associated with those. For ex- 
ample, the received signal is demodulated from optical 

20 input 1 01 -0W into an electronic signal, and then demul- 
tiplexed from a single ten-Gbit-per-second datastream 
in this example down to a parallel bus at a lower data 
speed. That parallel bus of signals then leaves module 
201 and goes into a processing module 202. Module 

25 202 contains an OC1 92 demultiplexer, which extracts a 
single 2.5 Gbit/second OC48 substream out of the 
OC192 stream and delivers a packet-over-SONET 
(POS) input to a framer 203-1, which is an industry 
standard off the shelf component. Likewise, module 202 

30 extracts the other three OC48 substreams and sends 
these to POS f ramers 203-2, 203-3, and 203-4 respec- 
tively. At this point there are four parallel 2.5 Gbit/sec 
SON ET streams,, one to each of four POS f ramers 203-1 
through 203-4, which extract from each OC48 stream 

35 the individual IP packets. POS framers 203-1 through 
203-4 first have to find the IP packets in the datastream 
and then have to extract the packets from the SONET 
continuous datastream. This is done on the four parallel 
OC48 streams. Once it has removed the packets from 

40 the SONET frame, each POS framer 203-1 through 

203- 4 delivers those packets to a facility ASIC 204-1 
through 204-4 respectively. 

The principal function of facility ASICs 204-1 through 

204- 4 is to send that information to an appropriate pack- 
45 et forwarding module (not shown in Fig. 2), in this case 

through an interface 103-0W consisting of four parallel 
interfaces for the four packet streams, or, if directed, to 
receive packets from an upstream neighboring facility 
ASIC on an interface 103-4W and switch 103-4W to 

so 103-0W in a protect mode. Otherwise, in a working 
mode of operation, a facility ASIC sends the information 
out through interface 103-OW, and information input on 
1 03-4W is directed through cascading protection bus in- 
terface 105-OW. The normal sequence is for a facility 

55 ASIC to take information from above and switch it below, 
letting the received traffic pass straight through onto in- 
terface 103-OW. All four of facility ASIC switches 204-1 
through 204-4 are ganged, such that they operate in par- 
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allel. With faster buses, faster framers, or faster facility 
ASICs, a single ASIC or bus, for example, could perform 
the above described functions instead of four required 
at the present state of technology. 
[0034] Figs. 5A and 5B are schematic diagrams illus- 
trating the functioning of a facility ASIC 204-1 in the nor- 
mal working mode and in the protection switch mode 
respectively. If there is no failed packet forwarding mod- 
ule (PFM), then a facility ASIC 204-1 is configured as 
shown in Fig. 5A, which illustrates protection switching 
in a single direction. In this case, a received signal on 
input interface 21 0 is sent through to output 21 3 , where- 
as the input protection bus 1 03-4W is connected to the 
output protection bus 105-OW. If a failure of an associ- 
ated packet forwarding module is detected, a microproc- 
essor (not shown) instructs facility ASIC 204-1 to go into 
the protection switch mode, illustrated in Fig. 5B. In this 
case, a signal from input 210 instead of going to output 
213 is switched to output protection bus 105-OW, input 
protection bus 1 03-4W is open and there is no signal 
available at output 213. This protection process in es- 
sence switches input 210 either to output 213 or down 
through output protection bus 105-OW, depending on 
whether facility ASIC 204-1 has been instructed to exe- 
cute a protection switch. The inverse directions going 
up the facility ASIC chain and out from the packet for- 
warding modules are essentially the reverse of Figs. 5A 
and 5B. Information traveling the reverse direction, that 
is from router 10 outbound on facility interface 1 02-0W, 
arrives from packet forwarding module 1 8-0 on interface 
104-0W. When there is no protection switch, the infor- 
mation goes directly through to facility module 12-0W, 
following the inverse of Fig. 5A. On the other hand, in 
the protection switch mode, the information is diverted 
through protection bus 1 06-OW. 

[0035] Referring again to Fig. 2, on the egress side 
facility ASIC 204-1 directs the information packets 
through output link 211 to Packet-over-SONET framer 
203-1 , which receives a packet, inserts it into a SONET 
frame, producing a 2.5 gigabit/second datastream or 
parallel bus equivalent, and sends that frame to OC192 
add/drop multiplexer 202. Multiplexer 202 combines 
four 2.5 gigabit/second streams from POS framers 
203-1 through 203-4, multiplexes them together into a 
1 0 gigabit/second datastream, and delivers them to op- 
tical transceiver 201. Transceiver 201 receives the 10 
gigabit/second stream, which is formatted as a parallel 
bus, and multiplexes it into a single datastream, which 
modulates a laser diode. This produces a SONET ten- 
gigabit/second optical format, which is transmitted 
through outbound optical facility interface link 102-0W. 
[0036] Fig. 3 is a block diagram illustrating information 
flow through a typical packet forwarding module 13-0 
(18-0). Facility ASICs 301-1 through 301-4 on the in- 
gress side receive packets from facility modules working 
and protect 11 -0W and 11 -OP through single links 
103-0W0 through 103-0W3. A principal function of facil- 
ity ASICs 301-1 through 301-4 on the ingress side is to 



select between the working and the protection facility 
modules, as represented by the information on, for ex- 
ample, incoming path 103-0W0 or 103-0P0. That selec- 
tion is made based on the standard SONET criteria for 
5 defining if one or both of those incoming facility modules 
is flawed or failed and also based on any detection of 
local errors or failures on working facility module 1 1 -0W 
or protect facility module 11 -OP. 

[0037] In the egress direction, a principal function of 

10 facility ASICs 301-1 through 301-4 is to duplicate the 
packet stream coming out of egress ASIC 302 and to 
send that packet stream out across both outgoing paths 
104-0W0 and 104-OPO to facility modules 12-OW and 
12-OP (see Fig. 2). 

15 [0038] Packet forwarding engines 306-1 through 
306-4 are devices that inspect the packet headers of all 
of the incoming packets received on any of the selected 
working or protect facility modules that are associated 
with this particular packet forwarding module 13-0 

20 (1 8-0). Based on the inspection of those headers, a de- 
termination of the intended destination of each packet 
can be made. The header information is stored by an 
ingress ASIC 304 in various queues and lists, which are 
used to determine for any given packet which output port 

25 of the router it should exit, when it should exit, and its 
relative priority. Actual packet data is stored by ingress 
ASIC 304 in an external RAM memory 305. Packet for- 
warding engine 306-1 through 306-4 also determines if 
any particular packet is intended for a local destination 

30 within this particular router and redirects it toward the 
main control processor of the router instead of transmit- 
ting it downstream out one of the output ports of the rout- 
er to a peer router across the network. 
[0039] Ingress ASIC 304, based on the states of the 

35 various queues that it maintains and based on the des- 
tination addresses of the various packets that are rep- 
resented by headers in those queues, sends requests 
through optical transceiver units 308-W and 308-P 
across optical link 310 (typically multimode ribbon fiber) 

40 to the central arbiter (not shown in Fig. 3). The central 
arbiter determines, based on ail of the packets that are 
being processed through the router in aggregate at any 
given time, which of the requests from a particular in- 
gress ASIC should be granted and when it should be 

45 granted for transmission across the optical switch. 
Grants of those requests return across optical link 310 
through transceivers 308-W and 308-P back to ingress 
ASIC 304. Ingress ASIC 304 uses that grant information 
to extract packets from memory 305 in the appropriate 

50 order and assembles them into chunk payloads. At the 
appropriate times ingress ASIC 304 sends those chunk 
payloads across channels 1 07-00 through 1 07-03 to in- 
ternal optics modules 14-0W0 through 14-NW1 (see 
Fig. 1B). 

55 [0040] On the egress side, information chunk pay- 
loads are received from the optical switch matrix indi- 
rectly through internal optics modules 17-OWO through 
17-NW1 (see Fig. 1B) across links 108-00 through 
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108-03 into an egress ASIC 302. Egress ASIC 302 18-N. 

reconfigures the chunks into packets and again stores [0044] In the egress direction, chunk payloads re- 
the packets in a memory 303 in the form of queues and ceived from internal optics modules 1 7 are broken down 
structures. Egress ASIC 302 subsequently reads those into their original packets by egress ASIC 302 (see Fig. 
packets out again into one of the four facility ASICs 5 3). The packets are stored in memory 303 and are then 
301-1 through 301-4. At the facility ASIC, each of those retrieved and delivered at the appropriate time to facility 
packet streams is duplicated and sent in tandem to both modules 12-0W and 12-OP. Each packet forwarding 
working and protect facility modules 1 2-OW and 1 2-OP. module 1 3 packages chunk payloads as described ear- 
[0041] A line control processor 307 is primarily re- lier and sends identical streams of chunk payloads to 
sponsible for controlling the facility protection switching 10 both working 1 and working 0 copies of the optical fabric 
function by examining the SONET error and failure in- via internal optics modules (lOMs) 14-0W0 through 
dications from facility modules 11 -0W and 11-0Pandal- 14-NW1 (see Fig. 1B). Working 0 copy of the optical 
so by analyzing the indications that facility ASICs 301-1 switch fabric includes internal optics modules 14-0W0 
through 301-4 develop from those incoming signals. and14-1W0, optical switch modules 15-1 through 15-6, 
The appropriate switching decisions are made in soft- 15 and internal optics modules 17-OWO and 17-1Wo] 
ware and logic and are then implemented by line control whereas working 1 copy of the optical switch fabric in- 
processor 307. eludes internal optics modules 14-0W1 and 14-1 W1 , op- 
[0042] Fig. 4 is a block diagram representing informa- tical switch modules 1 6-1 through 1 6-6, and internal op- 
tion flow through atypical internal optics module 14(17), tics modules 17-0W1 and 17-1W1. For example, IOM 
according to an embodiment of the present invention. 20 14-OWO and IOM 14-0W1 each receive simultaneous 
Internal optics module 14 receives chunk payloads of sequences of chunk payloads from each packet for- 
data via input links 107-00 through 107-04 from packet warding module 13 that is transmitting through those 
forwarding modules 13-0 through 13-N (see Fig. 3). An two lOMs. Similarly, on the egress side each packet for- 
internal optics ASIC 407 selects chunk payloads from warding module items 18-0 through 18-N (see Fig. 1C) 
those inputs based on grant information that comes 25 receives a simultaneous sequence of chunk payloads 
back from the central arbiter through each packet for- from lOMs 17-OWO and 17-0W1 , for example. In error- 
warding module 1 3-0 through 1 3-N . Internal optics ASIC free normal working operation of both optical switch fab- 
407 selects which inputs 1 07-00 through 1 07-04 will be rics, the simultaneous sequences of chunk data deliv- 
passed at any point in time to three MUXs 401-1 through ered to each packet forwarding module are identical. In 
401 -3 and out through three 12.5-gigabit-per-second 30 the event of a failure of any kind, either within a chunk 
transmitters 403-1 through 403-3 toward the optical or across multiple chunks on either copy zero or copy 
switch modules over single mode optical fiber links one of the optical switch fabric, the affected IOM is able 
109-1 through 109-3. Internal optics ASIC 407 is re- to detect that failure based on comparison of the re- 
sponsible for encapsulating the chunk payloads with the ceived FEC check sums with the calculated FEC check 
forward error correcting (FEC) headers and check sums 35 sums. When a failure on a particular chunk from either 
that guarantee that the chunks pass across the optical working zero or working one copy of the optical switch 
switchwithouterror,orthatiferrorsoccur,theyareeither fabric is detected, the IOM inserts a failure indication 
corrected or detected. MUXs 401-1 through 401-3 con- downstream toward PFMs 18. This forces PFM 18 to 
vert input parallel format data to higher bit rate serial select the error-free chunk data from the alternate copy 
data - 40 of the optical switch fabric. This can be done individually 
[0043] In the egress direction in Fig. 4, optical signals for each chunk payload delivered to a particular PFM. 
coming in over multimode optical fiber links 110-1 [0045] Referring again to Fig. 4, internal optics ASIC 
through 1 1 0-3 pass through 1 2.5-gigabit-per-second re- 407 detects any errors or failures of a given chunk on 
ceivers 404-1 through 404-3 and into three DEMUXs either copy zero or copy one of the switch fabric and 
402-1 through 402-3. Receivers 404-1 through 404-3 45 inserts appropriate failure indications downstream to- 
convert the data chunks from optical to electrical bits ward all of the packet forwarding modules connected to 
and DEMUXs 402-1 through 402-3 convert these from it. 

a serial bit stream to lower bit rate parallel bit streams. [0046] Referring again to Fig. 3, egress ASIC 302 re- 
Internal optics ASIC 407 compares the calculated FEC ceives those failure indications and selects on a chunk 
(forward error correction) check sums with the encoded so by chunk basis between either the copy zero or the copy 
check sums and determines if any errors have occurred one switch fabric. Only error-free chunk payloads from 
across the switch matrix, corrects those errors if possi- an unfailed switch fabric are inserted into memory and 
ble, and if not provides alarm and performance moni- subsequently retrieved and broken out into packets, 
toring information based on those errors. Internal optics which are then transmitted toward facility modules 
ASIC 407 then strips away the FEC coding from the 55 12-OW and 12-OP. 

chunks and passes the resulting chunk payloads from [0047] Fig. 6 is a flow diagram outlining the steps in- 

the demux channels out through links 108-00 through volved in performing an automatic PFM protection 

108-04 to packet forwarding modules 18-0 through switch, in accordance with an embodiment of the 
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present invention. The various associated apparatus 
modules are described above in connection with Figs. 
1A-1C, 2, 3, and 4. 

[0048] In step 601 of the flow diagram, a PFM fault is 
detected by a line shelf control module (LSCM), de- 
scribed in U.S. Application Serial Number 09/703,057, 
cited above, which is interconnected through a control 
network (CNET) with LCP 307 in PFM 13 (see Fig. 3). 
[0049] In step 602a the LSCM localizes and analyzes 
the PFM fault. For purposes of this discussion, the fault 
is assumed to occur in PFM 1 3-2 (18-2) and the protect 
PFM is assumed to be PFM 13-4 (18-4). It is further as- 
sumed that protect PFM 13-4 is operating in an "extra 
traffic" mode, such that it is carrying preemptable low 
priority traffic prior to the protection switch. This mode 
requires the most complex protection switch steps, 
which are sufficient to handle all other PFM protection 
switch cases. A similarly complex mode is the case in 
which the protect PFM is already protecting a working 
PFM when a higher priority working PFM fails. In the 
latter case, the protect PFM is already carrying data 
through the system and must be reconfigured for a dif- 
ferent working PFM. 

[0050] In step 602b the LSCM makes a protect deci- 
sion. In the present embodiment the LSCM manages 
the PFM protection switch process. 
[0051] In step 603 packet forwarding engines 306 on 
protect PFM 13-4(18-4) are configured to stop sending 
packets to ingress ASIC 304-4. This allows ingress 
ASIC 304-4 to empty its queues of all the currently buff- 
ered packets. If ingress ASIC 304-4 has a built-in way 
to reset its memory queues, then this step will also be 
performed on protect PFM 13-4 (1 8-4). This step will al- 
so prevent peer router messages (incoming from the fa- 
cility interfaces) from being sent to the master control 
processor (MCP) for protect PFM 13-4 (1 8-4). However, 
packet forwarding engines 306 on protect PFM 13-4 
(18-4) are still able to generate administrative packets 
to communicate with the MCP and flow control informa- 
tion can still be sent. This squelch operation could also 
be done at facility ASIC 301 on protect PFM 13-4(18-4). 
[0052] In step 604 egress lOMs 17 (see Fig. 4) are 
remapped to prevent any traffic addressed specifically 
for protect PFM 13-4 (18-4) from being received. This 
traffic is typically discarded. This is accomplished by dis- 
abling the output port on egress lOMs 17 connecting 
through lines 108 to egress ASIC 302-4 on protect PFM 
13-4 (18-4). 

[0053] In step 605, if egress ASIC 302-4 has a built- 
in way to reset its memory queues, then this step will 
also be performed on protect PFM 13-4 (18-4). 
[0054] In step 606 flow control for protect PFM 13-4 
(18-4) is cleared. This allows any module configured to 
send input to protect PFM 1 3-4 (1 8-4) to empty any buff- 
ers that may have been in a flow control "holding pat- 
tern." This is be accomplished by forcing a clear of flow 
control for all queues in egress ASIC 302-4 of protect 
PFM 13-4 (18-4). The updated flow control information 



is then distributed to the system through the normal flow 
control paths. 

[0055] In step 607 all traffic associated with the facility 
modules 11 -4W, 11 -4P (12-4W, 12-4P) connected to 
5 protect PFM 13-4 (18-4) is blocked. This squelching is 
accomplished by protect PFM LCP 307-4 informing 
POS framers 203 to insert path alarm indication signals 
(AIS). This step can be omitted if no facility modules ca- 
pable of interfacing with customers are associated with 

10 the protect PFM. Importantly, this action will prevent 
misconnects of packets to the wrong ports, which oth- 
erwise allow data to go out into the network through the 
facility modules associated with the protect PFM. 
[0056] In step 608 the ingress input and egress output 

15 on lOMs 1 4 (1 7) are disabled that lead to protected PFM 
13-2 (18-2). Inputs to protected PFM 13-2 (18-2) from 
FMs are blocked. This prevents protected PFM 13-2 
(18-2) from using system resources when it either has 
not failed or has failed uncontrollably. Shutting down the 

20 inputs to protected PFM 13-2 (18-2) allows the ingress 
and egress ASICs to clear their memory buffers. 
[0057] In step 609 the specific routing tables of pro- 
tected PFM 13-2 (18-2) are loaded into protect PFM 
13-4 (1B-4), along with any software state information, 

25 for example the current working/protect selection of fa- 
cility modules for protected PFM 13-2 (18-2), and 
weighted random early discard (WRED) provisioning, a 
TCP protocol packet discard policy that reduces con- 
gestion. 

30 [0058] Step 610 re-enables packet forwarding en- . 
gines 306 in protect PFM 1 3-4 (1 8-4) to resume forward- 
ing packets, although only idle packets should be re- 
ceived at this time. In other words, even though packet 
forwarding engines 306 are able to forward packets, 

35 they are not receiving any packets to forward, because 
lOMs 14(17) andassociated FMs still have their outputs 
blocked. 

[0059] Step 611 changes the identity of egress ASICs 
302-4 on protect PFM 13-4 (18-4) to virtualize protected 

40 PFM 1 3-2 (1 8-2). Also the flow control override in egress 
ASIC 302-4 is reversed to re-enable normal flow control 
operation using the new identity. 
[0060] In step 61 2 the LSCM using the CNET informs 
the arbitration shelf control module (ASCM) of the PFM 

^5 protection action, allowing the ASCM to configure the 
appropriate arbiter interface module (AIM) to route peer 
communication from the MCP to the appropriate PFM. 
This message received at the ASCM also configures the 
AIMs to clear the flow control settings for the unavailable 

so port(s) (in this case protect PFM 1 3-4) and resend to all 
the PFMs in the system. The LSCM also informs the 
MCP that the physical ports associated with protect 
PFM 1 3-4 (18-4) are now unavailable. The MCP, in turn, 
informs the entire system with a system table update 

55 that these ports are unavailable. 

[0061] In step 613the FMs are mapped as controlled 
by the LSCM to route the traffic and to signal protect 
PFM 13-4 (18-4) to route remote processor interface 
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(RPI) control to the FMs from protect PFM 13-4 (18-4) 
through the appropriate daisy chain bus 1 05 (1 06) using 
facility ASICs 204 (see Figs. 2 and 5A-5B). After this 
step, the protected PFM packets will start being forward- 
ed by protect PFM 13-4 (18-4). 5 
[0062] In step 614 egress lOMs 1 7 are mapped to al- 
low the received egress traffic normally directed to pro- 
tected PFM 1 3-2 (1 8-2) to go to protect PFM 1 3-4 (18-4). 
This allows egress ASIC 302-4 to start receiving the cor- 
rect traffic on protect PFM 13-4 (1 8-4), thus completing 10 
the protection switch at block 615. 
[0063] The steps to reverse the PFM protection switch 
are similar to the switch steps and are not detailed here. 
Of importance when reversing of PFM protection switch 
is to delay enabling the protect "extra traffic" until the '5 
working traffic is routed to the appropriate set of facility 
modules, to avoid misconnecting the data to the wrong 
ports. 

[0064] Referring again to Fig. 3, each packet forward- 
ing module 13 packages chunk payloads as described 20 
earlier and sends identical streams of chunk payloads 
to both working 1 and working 0 copies of the optical 
switch fabric via internal optics modules (lOMs) 14-OWO 
through 14-NW1 (see Fig. 1B) t which encapsulates the 
chunk payloads into chunks. Working 0 copy of the op- 25 
tical switch fabric (see Fig. 1B) includes internal optics 
modules 14-OWO and 14-1 W0, optical switch modules 
15-1 through 15-6, and internal optics modules 17-OWO 
and 17-1 W0, whereas working 1 copy of the optical 
switch fabric includes internal optics modules 14-0W1 30 
and 14-1W1 , optical switch modules 16-1 through 16-6, 
and internal optics modules 17-0W1 and 17-1 W1. For 
example, IOM 14-OWO and IOM 14-0W1 each receive 
simultaneous sequences of chunk payloads from each 
packet forwarding module 1 3 that is transmitting through 35 
those two lOMs. Similarly, on the egress side each pack- 
et forwarding module 18-0 through 18-N (see Fig. 1C) 
receives a simultaneous sequence of chunk payloads 
from lOMs 17-OWO and 17-0W1 , for example. In error- 
free normal working operation of both optical switch fab- 40 
rics, the simultaneous sequences of chunk data deliv- 
ered to each packet forwarding module are identical. In 
the event of a failure of any kind, either within a chunk 
or across multiple chunks on either copy zero or copy 
one of the optical switch fabric, the affected IOM is able 45 
to detect that failure based on comparison of the re- 
ceived FEC check sums with the calculated FEC check 
sums. When a failure on a particular chunk from either 
working zero or working one copy of the optical switch 
fabric is detected, the IOM inserts a failure indication so 
downstream toward PFMs 18. This forces PFM 18 to 
select the error-free chunk data from the alternate copy 
of the optical switch fabric. This can be done individually 
for each chunk payload delivered to a particular PFM. 
[0065] Note that while embodiments of the invention 55 
have been described in terms of two SONET standards 
namely OC48 and OC192, alternative implementations 
of router 10 having an appropriate facility module can 



operate under other standards. 

[0066] Embodiments according to the present inven- 
tion are designed to protect against all single fault oc- 
currences. Single faults include a single fault of a mod- 
ule, a single fault of a cable, or a single fault of a path. 
Accordingly, although some double faults are protected 
against, double faults generally lie beyond the scope of 
principal objects of the present invention and thus are 
not in general protected against. 
[0067] Although the present invention and its advan- 
tages have been described in detail, it should be under- 
stood that various changes, substitutions and altera- 
tions can be made herein without departing from the 
spirit and scope of the invention as defined by the ap- 
pended claims. Moreover, the scope of the present ap- 
plication is not intended to be limited to the particular 
embodiments of the process, machine, manufacture, 
composition of matter, means, methods and steps de- 
scribed in the specification. As one of ordinary skill in 
the art will readily appreciate from the disclosure of the 
present invention, processes, machines, manufacture, 
compositions of matter, means, methods, or steps, pres- 
ently existing or later to be developed that perform sub- 
stantially the same function or achieve substantially the 
same result as the corresponding embodiments de- 
scribed herein may be utilized according to the present 
invention. Accordingly, the appended claims are intend- 
ed to include within their scope such processes, ma- 
chines, manufacture, compositions of matter, means, 
methods, or steps. 



Claims 

1. A communication network comprising N substan- 
tially identical working line circuit cards, namely 
working line circuit cards w1, w2,..., wi,..., w(N-1), 
w(N) and M protect line circuit cards, namely protect 
line circuit cards p1, p2,..., pi, p(M-1),p(M), each 
substantially identical with said working line circuit 
card, wherein N and M is each a positive integer 
such that M is smaller than N; 

said line circuit cards having a working con- 
figuration such that each of said working line circuit 

cards w1 . w2 wi,.. M w(N-1 ), w(N) is connected in 

series into one of N respective separate data paths 

d1 , d2,..., di, , d(N-1 ), d(N) independent from any 

other of said working line circuit cards and having a 
protection switch configuration differing from said 
working configuration such that in at least one of 
said separate data paths di said working line circuit 
card wi is replaced by said substantially identical 
protect line circuit card pi. 

2. The communication network of claim 1 wherein 
none of said protect line circuit cards pi is connected 
into any of said data paths di in said working con- 
figuration. 
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3. The communication network of claim 1 wherein said 
M protect line circuit cards are a subset of said N 
working line circuit cards, such that said M protect 
line circuit cards pi are each connected into one of 
said data paths di in said working configuration and 5 
such that in said protection switch configuration 
said working line circuit card wi is replaced by said 
substantially identical protect line circuit card pi, 
wherein said working line circuit card wi and said 
substantially identical protect line circuit card pi are 10 
different line circuit cards. 

4. The comm unication network of claim 1 , further com- 
prising a router system, said router system incorpo- 
rating said working and said protect line circuit is 
cards. 

5. The communication network of claim 4, further com- 
prising an optical switch fabric incorporated into 
said router system, each of said data paths being 20 
interconnected with said optical switch fabric 
through an optical fiber cable. 

6. A method of router line card protection comprising: 

25 

providing N substantially identical working line 
circuit cards, namely working line circuit cards 
w1, w2,..., wi,..., w(N-1), w(N) and M protect 
line circuit cards, namely protect line circuit 
cards p1 , p2,..., pi, p(M-1), p(M), each sub- 30 
stantially identical with said working line circuit 
card , wherein N and M is each a positive integer 
such that M is smaller than N; 



20 

passed through faulty working line circuit card 
wi, such that said faulty working line circuit card 
wi is protected by protect line circuit card pi.. 

7. The method of claim 6 wherein none of said protect 
line circuit cards pi is connected into any of said data 
paths di in said working configuration. 

8. The method of claim 6 wherein said M protect line 
circuit cards are a subset of said N working line cir- 
cuit cards, such that said M protect line circuit cards 
pi are each connected into one of said data paths 
di in said working configuration and such that in said 
protection switch configuration said faulty working 
line circuit card wi is replaced by said substantially 
identical protect line circuit card pi, wherein said 
working line circuit card wi and said substantially 
identical protect line circuit card pi are different line 
circuit cards. 

9. The method of claim 8 wherein said protect line cir- 
cuit card pi carries preemptable low priority network 
traffic in said working configuration. 

10. The method of claim 6 wherein said working line cir- 
cuit cards and said protect line circuit-cards con- 
nected into said data paths perform information 
packet forwarding functions. 

11 . The method of claim 6 wherein said switching com- 
prises connecting and disconnecting said line cir- 
cuit cards into and out of said data paths using solid 
state switches. 

12. The method of claim 11, further comprising inter- 
connecting an optical switch fabric with each of said 
data paths through an optical fiber cable. 

13. The method of claim 11 wherein said switching fur- 
ther comprises: 

interrupting packet forwarding activity in proc- 
ess, if any, internal to said protect line circuit 
card pi; 

disabling all input and output network traffic 
through said protect line circuit card pi; 

disabling all input and output of network traffic 
through said faulty working line circuit card wi; 

loading specific routing tables of said faulty 
working line circuit card wi into said protect line 
circuit card pi; 

reenabling internal packet forwarding functions 
internal to said protect line circuit card pi; 



connecting said line circuit cards in a working 35 
configuration such that each of said working 
line circuit cards w1 , w2 : ..., wi,..., w(N-1), w(N) 
is connected in series into one of N respective 

separate data paths d1, d2 di d(N-1), d 

(N) independent from any other of said working 40 
line circuit cards; 

passing network traffic in parallel through said 
N working line circuit cards in said working con- 
figuration; 45 

if a fault is detected in a working line circuit card 
wi, then switching from said working configura- 
tion to a protection switch configuration differ- 
ing from said working configuration such that in 50 
at least one of said separate data paths di said 
faulty working line circuit card wi is replaced by 
said substantially identical protect line circuit 
card pi; and 

55 

passing network traffic in said protection switch 
configuration through protect line circuit card pi 
that in said working configuration previously 
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virtualizing the identity of said faufty working 
line circuit card wi into said protect line circuit 
card pi; and 

rerouting all input and output network traffic that 
previously passed through said faulty working 
line circuit card wi, such that said network traffic 
is redirected through said protect line circuit 
card pi. 
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